SplitNet: A Dynamic Hierarchical Network Model

Graph Properties and Retrieval We investigate the information that is contained in the structure of a topology preserving neural network. We consider a topological map as a graph G, propose certain properties of the structure and formulate the respective expectable results of network interpretation. The scenario we deal with is the nearest-neighbor approach to classification. The problems are to find the number and positions of neurons that is useful and efficient for the given data and to retrieve a list L of m nearest neighbors (where r-r1 is not necessarily known in advance) for a presented query q that is to be classified. First, we assume a complete storage of data records in the graph G, i.e. each data record is represented by a neuron, and a perfect topology preservation, which means that an edge between neurons Ni and Nj is in G iff Ri n Rj # 8 where Ri denotes the Voronoi region of node 2ri. Thus, the graph corresponds to the Delaunay triangulation of the nodes in G. For this situation, we can formulate an algorithm that is complete at any stage of its incremental retrieval. Considering complete storage but imperfect topology preservation, we deal with a subgraph of the above mentioned Delaunay graph. We use a topographic function (Villmann et al. 1994) to measure the topology preservation and describe it by the characteristic number t+ which is the size of the largest topological defect. We can reformulate the previous retrieval algorithm for this case and again its completeness can be shown. The efficiency of the algorithm depends exponentially on the value oft+ , so a good topology preservation of the network is needed. However, if incomplete storage is investigated, we can show that we have to restrict the neuron distribution in the data space. If we use a quantizing method, we can minimize the probability of incompleteness of the retrieved list of nearest neighbors to a given query. Guided by these insights, we developed the SplitNet model that provides interpretability by neuron distribution, network topology and hierarchy. The SplitNet Model SplitNet is a dynamically growing network that creates a hierarchy of topologically linked one-dimensional Kohonen chains (Kohonen 1990). Topological defects in the chains are detected and resolved by splitting a chain into linked parts, thus keeping the value of t+ fairly low. These subchains and the error minimizing insertion criterion for new neurons (similar to the one presented in (Fritzke 1993)) provide the quantization properties of the network.